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® Distributed data processing system. 

® A distributed data processing system includes a distributed resource manager which detects dependencies 
between transactions caused by conflicting lock requests. A distributed transaction manager stores a wait-for 
graph with nodes representing transactions and edges the nodes and representing dependencies between the 
transactions. Each edge is labelled with the identities of the lock requests that caused the dependency. The 
distributed transaction manager propagates probes through the wait-for graph, to detect cyclic dependencies, 
indicating deadlock. A deadlock message is then sent to the resource manager identifying a particular lock 
request as a victim for detection to resolve the deadlock. 
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Background to the Invention 

This invention relates to distributed data processing systems. 

Such a system generally includes a number of resources which may be accessed by a number of user 
5 processes. Access to the resources may be controlled by a lock management mechanism. When a user 
process requires to access a resource, it requests a lock for that resource. If the lock cannot be granted 
immediately, it is queued. 

In such a system, dependencies can occur between transactions; that is, one transaction may be held 
up, waiting for another to release a lock on a particular resource. If the dependencies become cyclic all the 
10 transactions in the cycle are held up waiting for each other. This is referred to as a deadlock. 

To overcome this problem, deadlock detection mechanisms have been proposed. When a deadlock is 
detected, one of the transactions in the cycle is selected as a "victim", and is aborted so as to break the 
deadlock. 

The object of the present invention is to provide an improved deadlock detection and resolution 
75 mechanism. 

Summary of the Invention 

According to the invention there is provided a distributed data processing system comprising a plurality 
20 of processing units wherein the processing units contain: 

(a) a plurality of resources. 

(b) a plurality of user processes performing transactions which require access to the resources, and for 
producing lock requests, requesting locks on the resources for said transactions. 

(c) a resource manager responsive to said lock requests from the user processes, for maintaining a 
25 queue of lock requests that cannot be immediately granted, and for detecting dependencies between 

transactions caused by conflicting lock requests. 

(d) means for storing a wait-for graph comprising a plurality of nodes representing transactions and a 
plurality of edges interconnecting the nodes and representing said dependencies between the transac- 
tions, each edge being labelled with the identities of the lock requests that caused the dependency, and 

30 (e) means for detecting a cyclic chain of dependencies in the wait-for graph and, upon detection of a 
cyclic chain of dependencies, for sending a deadlock message to the resource manager identifying a 
particular lock request as a victim for deletion to resolve the deadlock. 
It can be seen that the invention allows deadlock to be detected and resolved on the basis of individual 
locks within transactions. As a result, deadlock can be restricted to a lock within a transaction, and the user 
35 does not have to abort the whole transaction. Deadlock is detected, not on the basis of time-outs, but by 
efficiently transmitting information on dependencies between locks. 

Brief Description of the Drawings 

40 Figure 1 is an overall block diagram of a distributed data processing system in accordance with the 

invention. 

Figure 2 is a schematic diagram of a global wait-for graph used by a deadlock detection mechanism. 

Figure 3 is a schematic diagram showing the way in which the wait-for graph is distributed between 
individual transaction managers. 
45 Figure 4 is a block diagram showing the various components of the system and the data structures 

used by them. 

Description of an Embodiment of the Invention 

50 One distributed data processing system in accordance with the invention will now be described by way 

of example with reference to the accompanying drawings. 

Referring to Figure 1. the system comprises a number of processing units 10, interconnected by a 
communications network 1 1 which allows messages to be passed between the processing units. 

A number of client programs 12 are distributed amongst the processing units. A client program may, for 
55 example, be a database program or some other user application. The clients access a number of resources 
which may, for example, be data items. 

Access to the resources is controlled by a distributed lock manager (DLM) which consists of a set of 
agents 13 and servers, distributed amongst the processing units. The servers include process managers 
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(PM) 14, session managers (SM) 15, resource managers (RM) 16 and transaction managers (TM) 17. 
Agents 

5 Each agent represents a set of clients, ie it acts as an interface between those clients and the 

distributed lock manager. Each client is represented by at most one agent. For example, in a UNIX system, 
the agent may be the UNIX kernel and a client may be a UNIX process. 

Process Managers 

10 

The distributed lock manager views the clients as a set of processes. More than one process may be 
associated with each client. Each process is owned by a particular agent, and each process is identified by 
a process identifier that is unique within the lock manager. 

The process managers PM keep a record of the processes that have been created, and their owners. 
75 Each process manager maintains information about a subset of processes. 

Session Managers 

The session managers keep a record of the sessions that have been opened, and their owners. 

20 A session is defined as the unit of processing to which locks may be granted. A session is owned by 

one or more processes of the same agent. Processes may open and close sessions and the same session 
may be opened by different processes. Locks that are owned by a session may thus be accessed by 
different processes. Clients identify a session by a session name supplied by the client when the session is 
opened. The lock manager identifies a session by a session identification which is unique within the lock 

25 manager. 

Transaction Managers 

A transaction is defined as the unit in a process that requests locks for resources. A transaction may be 
30 distributed over a plurality of sessions, not necessarily all owned by the same process. Each transaction 
has a transaction name provided by the client. 

Referring to Figure 2, the transaction managers TM collectively maintain a directed graph structure, 
referred to as the wait-for graph (WFG), which indicates wait-for dependencies between transactions. The 
WPG consists of a set of nodes, interconnected by edges. The nodes of the WFG represent transactions, 
35 and the edges represent dependencies between transactions. Specifically, node T(i) represents transaction 
i, and an edge directed from a first node T(i) to a second node T(j) indicates that the transaction 
represented by the first node is waiting for the transaction represented by the second node to release a 
lock on a particular resource. The transaction that is waiting is referred to as the tail transaction, and the 
transaction that is waited for is referred to as the head transaction. 
40 Referring now to Figure 3, the WFG is distributed between the TMs such that each TM stores a sub- 

graph of the WFG. The nodes and edges of the WFG are distributed between the TMs in accordance with 
some predetermined mapping function that maps transaction identifiers on to the transaction managers. In 
this example, the node T{i), and all edges directed from that node, are held in transaction manager p, 
where: 

45 

p = i modulo n 

and n is the total number of TMs in the system. 

In the example of Figures 2, 3 it is assumed that there are 3 TMs, so that n = 3. Hence, nodes T(0), T- 
50 (3), T(6) etc are stored in TMO, nodes T(1), T(4). T(7) etc stored in TM1, and nodes T{2), T(5), T(8) etc are 
stored in TM2. 

Each edge in the WFG is labelled with an edge value. The edge value indicates between which two 
locks of both transactions the dependency exists. There can be multiple edges between the same two 
nodes, since more than one lock of one transaction can wait for more than one lock of another transaction. 
55 The edge value uniquely identifies each edge between two transactions. 

It can be seen that any cyclic chain of edges in the WFG indicates a deadlock situation in which a 
number of transactions are held up waiting for each other to release locks. For example, in Figure 2, it can 
be seen that transactions 17, 3, 6, 7, 1,0 and 5 are cyclically dependent on each other and hence there is a 
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potential deadlock. Deadlock cycles are detected by the TMs by propagating special messages, referred to 
as probes, through the WFG. A probe is Initiated whenever an edge is inserted in the WFG, and carries the 
value of this edge (referred to as the initiating edge) with It while propagating through the WFG. Each node 
has a priority associated with it, determined by a timestamp value, and probes are propagated through the 
5 WFG according to a priority scheme, a probe being forwarded along an edge only if the head transaction of 
that edge has a lower priority than the tail transaction of the probe-initiating edge. This reduces the number 
of probe messages required. 

As the probe traverses the WFG. copies of It are stored in each node of the WFG through which it 
passes. If there exists more than one path through which a probe can reach a node, the same probe may 
w be received more than once. Only the first received copy of the probe is stored in the node. Any 
subsequently received copies increment a count in the stored copy of the probe, but are not propagated 
any further through the WFG. If a probe returns to its initiating edge then a deadlock is detected. The 
deadlock is resolved by selecting one of the locks in the cycle as a "victim" to be cancelled. 

When an edge is deleted from the WFG an antiprobe message is initiated. The antiprobe follows the 
IS same route as the probe that was initiated when this edge was inserted, but reverses. that probe's actions. If 
the count value for a probe stored at a node is greater than one, the count is decreased by one and the 
antiprobe is not further fonwarded. If the count value is one, the stored probe is removed from the node and 
the antiprobe is forwarded. 

It can be shown that by using this priority scheme a deadlock cycle is detected by exactly one probe 
20 that is initiated from an edge in the cycle. 

Resource Managers 

The resource managers RM keep a record of the locks that have been granted for resources. Each lock 
25 has one of the following lock modes which indicate the required level of access to a resource: 
Null (N): This mode grants no access to the resource. 

Intention-shared (IS): This mode grants read access to the resource and allows sharing of the resource 
with other readers. Writers are allowed to the resource. 

Intention-exclusive (IX): This mode grants write access to the resource and allows sharing of the 
30 resource with other writers. 

Shared (8): This mode grants read access to the resource and allows the resource to be shared with 
other readers. No writers are allowed access to the resource. 

Shared and intention-exclusive (SIX): This mode grants write access to the resource and allows the 
resource to be shared with intention-shared mode readers. No other writers are allowed access to the 
35 resource. 

Exclusive (X): The exclusive mode grants write access to the resource and prevents the resource from 
being shared with any other readers or writers. 

Locks that can be shared with other locks are said to have compatible lock modes. Table 1 shows the 
compatibility of the lock modes. 

40 
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TABLE 1 

20 

Thus, it can be seen for example that the intention-shared (IS) mode is compatible with all the other 
modes except the exclusive (X) mode. 

Clients may open and close locks on resources and convert the mode of an opened lock to a different 
mode. A lock is always opened in the NULL mode which is. by definition, compatible with all other locks. 

25 However, a lock in NULL mode is not effective until it is converted to a more restrictive mode. 

If a lock conversion changes a lock to a mode that is incompatible with other opened locks, the lock 
conversion has to wait and the lock is queued on a waiting queue. When a lock conversion is granted or 
cancelled, or a lock is closed, other lock conversions that are queued on the waiting queue may be granted. 
The RM then checks whether any new wait-for dependencies have been created, or any existing 

30 dependencies removed. A dependency may exist between a lock on the waiting queue and either a granted 
lock, or another lock further ahead of it on the waiting queue. If a dependency has been created, the RM 
sends a message to the appropriate TM, instructing it to insert a new edge in the WFG. Conversely, if a 
dependency has been removed, the RM sends a message to the appropriate TM. instructing it to delete the 
corresponding edge of the WFG. 

35 Each resource has an associated value block. The first time a lock is opened for a resource, the lock 
manager associates a value block with the resource. The lock manager maintains the resource and its value 
block until all locks on the resource have been closed. A value block" contains an value block status and a 
value block value. The status is either valid or invalid. The value block value contains some client supplied 
value, provided that the value block status is valid. By default the resource value block status is initialised to 

40 invalid. Clients may read or write the value block of a resource, depending on the mode of the lock they 
have opened on the resource. 

Communication Between Clients and Agents 

45 Referring now to Figure 4, each client 12 has an associated event queue 40. A client communicates 
with the distributed lock manager by sending function calls to the agent 13 that represents it. The agent 
may return information to the client synchronously in response to the function call. Alternatively, the agent 
may respond asynchronously by posting an event on the client's event queue. Clients poll events from their 
event queues. 

50 

Data Structures 

The agents and servers in the distributed lock manager maintain a number of data structures, referred 
to herein as pools. Each pool consists of a set of tuples, each tuple consisting of a set of data members. 
55 Each process manager RM maintains a process pool 41, each session manager SM maintains a session 
pool 42. each resource manager RM maintains a resource pool 43, a transaction pool 44, and a lock pool 
45, each transaction manager TM maintains a node pool 46. an edge pool 47 and a probe pool 48, and 
each agent maintains an agent process pool 49, an agent session pool 50, an agent lock pool 51 , and a 
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pending queue 52. 
Process Pool 

5 The process pool consists of a. set of process tuples, each or which contains information about a 

particular process. Each tuple comprises the following members, 
owner the identity of the agent that owns the process, 
prcid: the process identifier. 

10 Session Pool 

The session pool consists of a set of session tuples, each of which contains information about a 
particular session. Each session tuple comprises the following members, 
owner: the identity of the agent that owns the session. 
75 sesid: the identity of the session. 

sesname: the session name supplied by the agent. 

prcid set: a set of identifiers, identifying the processes involved in the session. 

Resource Pool 

20 

The resource pool consists of a set of resource tuples, each of which contains information about locks 
that have been opened for a particular resource. Each resource tuple comprises the following members, 
rscname: a resource name that uniquely identifies the resource tuple in the resource pool. This name is 
supplied by the client. 

25 Ickid set: a set of lock identifiers for the locks that are currently open for this resource. A subset of this 

set may be queued on the waiting queue of the resource. 

waiting queue: a set of lock identifiers for the locks that are currently on the waiting queue of the 

resource. 

viblk: the value block of the resource. 

30 

Transaction Pool 

The transaction pool consists of a set of transaction tuples, each of which holds information about a 
particular transaction. Each transaction tuple comprises the following members. 
35 traname: a transaction name that uniquely identifies the transaction tuple in the transaction pool. This name 
is supplied by the agent. 

'ckid set: a set of lock identifiers specifying the locks that are owned by this transaction. 

Lock Pool 

The lock pool consists of a set of lock tuples, each of which holds information about a particular lock. 
Each lock tuple comprises the following members, 
owner: the identity of the agent that owns the lock. 

Ickid: a lock identification that uniquely identifies the lock tuple in the lock pool. This identification is unique 
45 within the RMs. 

sesid: the identifier of the session that owns this lock, 
rscname: the name of the resource controlled by this lock, 
traname: the name of the transaction that requested this lock. 

status: this is either "granted" (not queued on waiting queue) or "waiting" (queued on waiting queue). 

50 granted mode: the lock mode in which the lock has been granted. 

waiting mode: if a conversion has been requested for the lock, this holds the mode to which the lock is to 

be converted. 

key: if the lock status is waiting, this identifies a corresponding pending tuple on the pending queue of the 
agent that owns this lock tuple. 

55 
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Node Pool 

The node pool consists of a set of node tuples, each of which holds information about a particular WFG 
node. Each node tuple comprises the following mennbers. 
5 traname: the name of the transaction represented by this node, 
stamp: a timestamp value, unique within the TMs. 
card; the cardinality of the node tuple, ie a count value. 

Edge Pool . 

The edge pool consists of a set of edge tuples, each of which holds information about a particular edge 
in the WFG. Each edge tuple comprises the following members, 
owner the identity of the RM that owns this edge tuple. 

edge_label: this uniquely Identifies the edge tuple in the edge pool, and comprises a label (11, t2, II, 12), 
where tl and 12 are the tail and head transactions of the edge, and 11 and 12 are the locks in t1 and t2 that 
caused the wait-for dependency represented by this edge, 
card: the cardinality of the edge tuple, ie a count value. 

Probe Pool 

20 

The probe pool consists of a set of probe tuples, each of which contains infornnation about a stored 
probe. Each probe tuple comprises the following members, 
owner: the TM that owns this probe tuple. 

init edge: the label of the edge in the WFG that initiated this probe. 

25 stamp: a timestamp for the tail transaction of the initiating edge, 
node: tfie node at which this probe is stored, 
card: the cardinality of the probe tuple, ie a count value. 

Agent Process Pool 

30 

The agent process poo! consists of a set of process tuples, each with the following members, 
owner: this identifies the PM that owns the process. 

prcid: the process identification, which uniquely identifies the process tuple in the process pool, 
client: this identifies the client associated with this process. 

35 

Agent Session Pool 

The agent session pool consists of a set of session tuples, each with the following members, 
owner this identifies the SM that owns this session. 
40 sesid: the session identification, which uniquely identifies the session tuple in the session pool, 
sesname: a client-supplied name for the session. 

prcid set the identifiers of all processes that have opened the session. 

Agent Lock Pooi 

45 

The agent lock pool consists of a set of lock tuples, each of which has the following members, 
owner this identifies the RM that owns this lock. 
Ickid: this uniquely identifies the lock tuple In the lock pool, 
sesid: this identifies the session that owns this lock, 
so rscname: this identifies the resource for which the lock is queued, and is supplied by the client, 
traname: this identifies the transaction for which this lock is queued, and is supplied by the client. 

granted mode: this Identifies the lock mode in which the lock is currently granted. 

viblk: the resource value block, 
uhandle: a user handle for the lock. 

55 
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Pending Queue 

The pending queue holds pending requests from this agent to the servers. The pending queue is an 
ordered lisi of pending tuples. Each pending tuple has the following members. 
5 owner: ihis identifies the server for which the request is pending. 

key: this is an identification for the pending tuple, which uniquely identifies the tuple on the agent's pending 
queue. 

client: this identifies the client on whose behalf the agent made the request, 
uhandle: a user handle for the request. The meaning of this value is private to the client. 
JO request: the request message that has been sent to the server. 

Operation 

The operation and interaction of the various components of the distributed lock manager will now be 
75 described. 

Create Process 

When a client C toquttes to create a process, it sends a call Im create process(uhandle) to the agent 

20 Ak that represents C 

When it receivo?; this call, the agent Ak randomly selects a process manager PMi, sends a 
NEWPROCESS(key) mos.-^agfi to PMi, and queues a pending tuple (Pr\/li.key,C,uhandle.message) in its 
pending queue. Ak thnn r,-jturns a LM_SUCCESS status to the client. 

When PMI receives the NEWPROCESS{key) message from Ak. it constructs a process identification 
" 25 prctd and inserts a process tuple (Ak.prcid) in its process pool. PMi then replies with a 
NEWPROCESSCOMPLETE(key.prcid) message to Ak. 

When Ak receives the NEWPROCESSCOMPLETE{key.prcid) message from PMi, it removes the related 
pending tuple from its pending queue and inserts the process tuple (PMi.prcid) in its process pool. Ak then 
posts a Im create_process(uhandle,LM SUCCESS.prcid) event on C's event queue. 

30 

Delete Process 

When a client C requires to delete a process, it sends a call Im delete_process(uhandle,prcid) to the 

agent Ak that represents C. 

35 When Ak receives this call, it looks up the process tuple in its process pool. If it cannot find the tuple, 

the process identifier prcid must be invalid and so an LM INVPRCID status is returned to C. If it finds the 

tuple, Ak checks whether C is the owner of the tuple and, if it is not, returns a LM NOTOWNER status to 
C. ~ 

Assuming that Ak found the tuple and C is the owner, Ak then sends a DELPROCESS(prcid) message 
40 to the process manager PMi that owns the process tuple identified by prcid. Ak then removes the process 
tuple from its process pool. Ak then closes all sessions that are opened by this process, and returns a 
LM_SUCCESS status to C. 

When PMi receives the DELPROCESS(prctd) message from Ak, it looks up the process tuple that is 
identified by prcid and removes it from its process pool. 

45 

Open Session 

When a client C wishes to open a new session, it sends a call Im open session- 

(uhandle.prcid.sesname) to the agent Ak that represents the client C. 
50 When Ak receives this call, it selects the session manager SMI that maintains the session tuple 
identified by sesname, in accordance with a predetermined mapping function that maps session names on 
to session managers. Ak then sends an OPENSESSION(key,prcid,sesname) message to SMi and queues a 
pending tuple (SMi,key,C,uhandle.message) in its pending queue. Ak then returns a LM_SUCCESS status 
to C. 

55 When SMi receives the OPENSESSION(key,prcid,sesname) message from Ak, it looks up the session 

tuple identified by sesname. If SMi cannot find the session tuple, it constructs a new session tuple with Ak 
as owner and adds it to its session pool. 
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!f the tuple is found, SMi checks whether the session tuple is owned by Ak. If it is not owned by Ak, 
SMi returns the message OPENSESS!ONCOMPLETE(key,sesid,LM_NOTOWNER) to Ak. When it receives 
tills message, Ak removes the related pending tuple from its pending queue, and posts a 

lrn_upuM_session(uhandle,LM NOTOWNER.sesid) event on C's event queue. 

i. W a tuple owned by Ak was found, or a new tuple was created, SMI then adds prcid to prcid set In the 

soss.on tuple, and replies with an OPENSESSIONCOMPLETE(key,sesid.LM_SUCCESS) message to Ak. 

V/hon Ak receives the 

OP[:NSCSSIONCOMPLETE(key,sesld,LM_SUCCESS) nnessage from SMi, It removes the related pending 
tuple Itom tts pending queue. Ak then looks up the session tuple in Its session pool. If It cannot be found, a 

to session tuple Is constructed and added to the pool. Finally, Ak adds prcid to the prcid set In the session 

tuple and posts a 

im open scDSton(uhandle,LM SUCCESS, sesid) event on C's event queue. 

Close Session 

15 

V/hon a client C requires to close a session, it sends a call Im close sesslon{uhandle, prcid, sesId) to 

the agent Ak that represents C. 

Ak looks up the session tuple in its session pool. If It cannot find the tuple, the sesId must be invalid 
and so a LM_tNVSESID status Is returned to C. tf C Is not the owner of the session tuple, Ak returns a 
20 LM__NOTOWNFR status to C. 

If a tuple nwnnd by C was found, Ak sends a CLOSESESSION(prcld,sestd) message to SMI. where SMI 
Is the session man^^ger that owns the session tuple identified by sesld. Ak then removes prcid from the 

prcid set m the session tuple. If prcid set becomes empty as a result of this. Ak closes all locks that are 

owned by the session. Ak then returns an LM SUCCESS status to C. 

25 When SMi receives the CLOSESESSION(prcid.sesld) message from Ak. It looks up the session tuple in 

its session pool and removes prcid from the prcid set of the session tuple. If prcid set becomes empty 

as a result of this, the session tuple is removed from the session pool. 

Open Lock 

30 

When a client C requires to open a lock, It sends the call Im open_lock- 

(uhandle, sesid, rscname.traname) to the agent Ak that represents C, where rscname Is the name of the 
resource on which the lock is requested, and traname Is the name of the transaction that requires the lock. 

When Ak receives this call, it selects the resource manager RMI that maintains the resource tuple 
35 identified by rscname, according to a predetermined mapping function that maps resource names on to the 
resource managers. Ak sends an OPENLOCK(key, sesld, rscname.traname) message to RMI and queues a 

pending tuple (RMI.key.C.uhandle, message) in its pending queue. Ak then returns a LM SUCCESS status 

to C. 

When RMI receives the OPENLOCK{key,sesid.rscname.traname) message from Ak, It looks up the 
40 resource tuple identified by rscname in Its resource pool, tf it cannot be found, the resource tuple is 
constructed and added to the resource pool. The resource tupie Is constructed with an invalid value block. 
RMI then looks. up the transaction tuple identified by traname in its transaction pool. If It cannot be found, 
the transaction tuple Is constructed and added to the transaction pool. RMi then constructs a lock tuple and 
adds it to the lock pool. RMi also adds the lock to the lock set of the resource tuple and to the lock set of 
45 the transaction tuple. RMi then sends an OPENLOCKCOMPLETE(key,lckid,vlblk) message to Ak. The 
message Includes the resource value block viblk. 

When Ak receives the OPENLOCKCOMPLETE(key,lckid.vlblk) message from RMi, it removes the 
related pending tuple from its pending queue. Ak then constructs a lock tuple and adds the lock tuple to Its 
lock pool. Ak then posts a Im open_!ock(uhandle,LM SUCGESS.IckId) event on C's event queue. 

50 

Close Lock 

When a client C requires to close a lock, it issues the call Im close lock(uhandle,lckid) to the agent 

Ak that represents C. 

55 When Ak receives this call, it looks up the lock tuple In its lock pool, if It cannot find the lock tuple. Ickid 

must be Invalid and so a LM INVLCKID status is returned to C. Otherwise Ak proceeds with the following 

steps. 
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Ak sends a CLOSELOCK{lckid) message to the resource manager RMi that owns the lock tuple 

identified by Ickid. Ak then removes the lock tuple from its lock pool and returns a LM SUCCESS status to 

.C. Any pending conversion or cancel remains on Ak's pending queue. 

When RMi receives the CLOSELOCK(lckid) message from Ak. it looks up the lock tuple in its lock pool. 
5 and accesses the related resource and transaction tuples in its resource and transaction pools. RMi then 
checks whether any transaction dependencies have ceased as a result of closing the lock. If so, RMi sends 
DELEDGE messages to the appropriate TMs instructing them to delete the corresponding edges from the 
WFG. RMi then removes the lock from the set of locks in the transaction tuple and from the set of locks in 
the resource tuple and deletes the lock tuple from the lock pool. If the lock was on the waiting queue, Ak 
70 has an outstanding lock conversion, and so RMi sends a 

CONVERTUPCOMPLETE(key,lckid,vlblk,LM_CLOSED) message to Ak. 

RMi then checks if any other locks in the waiting queue of the resource can now be granted. A lock 
from the head of waiting queue can be granted if the requested lock mode is compatible with the other 
granted locks. If the lock can be granted, it is removed from the waiting queue, and the lock tuple is 
75 updated by setting its granted mode to the requested conversion mode and its status to "granted". RMi 
then sends a CONVERTUPCOMPLETE(key.lckid.vlblk,LM_GRANTED) message to the lock owner agent of 
each newly granted lock. That agent then acts as described below for the case of an upward lock 
conversion. 

If Ak receives a CONVERTUPCOMPLETE(key,lckid,v!blk.LM_CLOSED) message from RMi. it looks up 
20 the lock tuple in its lock pool, but in this case, it cannot find the tuple (since it has already removed the lock 
tuple from the lock pool). Ak then removes the related pending tuple from the pending queue and posts a 
Im convert lock(uhandle,LM CLOSED.vlblk) event in C*s event queue. 

Lock Conversion 

25 

Two types of lock conversion are distinguished, namely upward lock conversion and downward lock 
conversion, according to the following table: 
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down 


up 


down 


up 


up 




SIX 


down 


down 


down 


down 


down 


up 


45 
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down 


down 


down 


down 


down 


down 



TABLE 2 



50 Thus, it can be seen for example that converting a lock from intention-shared (IS) to exclusive (X) is 
classified as an upward conversion, since the new mode is more restrictive than the old mode. 

An upward lock conversion brings a lock in a more restrictive lock mode and such a conversion is only 
granted if (1) there are no other conversions outstanding on the resource and "(2) the requested mode is 
compatible with the other granted lock modes. A downward lock conversion brings a lock into a less 

55 restrictive lock mode and is always immediately granted. The agent can determine if a conversion is 
upward or downward from the lock Information that it keeps in its lock pool and from the requested 
conversion mode. 
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The essential difference between the two is that a downward lock conversion does not require any 
connpletion message, whereas an upward lock conversion does. On a downward lock conversion, the agent 
updates its lock tuple prior to sending the convert lock message. On an upward lock conversion, the agent 
updates its lock tuple when it receives a reply from the RM. A downward lock conversion is always 
5 immediately granted, whereas an upward lock conversion potentially has to wait before it is granted. 

When a client 0 requires to convert a lock, it issues the call Im convert lock- 

(uhandle,lckid,mode,vlblk, notify) to the agent Ak that represents C. 

When Ak receives this call, it looks up the lock tuple in its lock pool. If it cannot find the tuple, Ickid 

must be invalid and so a LM INVLCKID status is returned to the caller. If the tuple can be found, but 

70 another conversion is pending for the lock, a LM ALREADY status is returned to the caller. 

If the tuple was found, and no other conversion was pending, Ak proceeds as follows. First, Ak checks if 
the conversion is upward or downward by comparing the requested lock mode with the currently granted 
lock mode. 

If the conversion is upward, Ak sends a 
15 CONVERTUP{key, Ickid, mode) message to the resource manager RM\ that owns the lock tuple: Ak then 

queues a pending tuple (RMi,key,C,uhandle,message) in its pending queue and returns a LM SUCCESS 

status to the caller. 

When RMi receives the CONVERTUP(key,lckid,mode) message from Ak, it looks up the lock tuple and 
the related resource tuple for which the lock is queued. RMi then checks if the waiting queue of the 
20 resource is empty and the requested lock mode is compatible with the other granted locks. If so, RMi can 
grant the conversion immediately, and so it changes the granted mode in the lock tuple to the requested 
conversion mode, and sends a 

CONVERTUPCOMPLETE(key.lckid.vlblk.LM_GRANTED) message to Ak. If on the other hand RMi cannot 
grant the conversion immediately, the lock tuple Is queued by adding its lock identifier to the waiting queue 

25 of the resource tuple. The lock tuple is then updated by setting its status to "waiting", its waiting mode to 
the requested conversion mode, and its key value to the key value from the message. RMi then checks 
whether any new transaction dependencies have been created as a result of queuing the lock conversion. If 
so, INSEDGE messages are sent to the appropriate TMs to instruct them to insert corresponding edges into 
the WFG. 

30 When Ak receives a 

CONVERTUPCOMPLETE(key,lckid.vlblk,LM_GRANTED) message from RMi, it checks if there is a pend- 
ing cancel for the lock in its pending queue. If so. this cancel has failed and a lm_cancel_lock- 

(uhandle,LM GRANTED) event is posted on C's event queue. Ak then looks up the lock tuple in its agent 

lock pool. If Ak finds the lock tuple, it updates the lock tuple by setting its granted mode to the requested 

35 conversion mode and its value block to the returned value block from the message. Finally, Ak removes the 
related pending tuple from its pending queue and posts a 

Im convert lock(uhandle,LM GRANTED,vlblk) event in C's event queue. 

If on the other hand Ak cannot find the lock tuple, this means that the agent has closed the lock and the 
lock is closed or is being closed on server side. Ak therefore removes the related pending tuple from the 

40 pending queue and posts a Im convert lock(uhandle,LM GRANTED.vlblk) event in C's event queue. 

If the lock conversion is downward, Ak proceeds as follows. Ak sets the granted mode in the lock tuple 
to the requested conversion mode. Ak then sends a 

CONVERTDOWN{lckid.mode,vlblk,) message to the resource manager RMi that owns the lock tuple, and 
returns a LM_SUCCESS status to the caller. 

45 When RMi receives the CONVERTDOWN{lckid,mode,vlblk) message from Ak, it looks up the lock tuple 
and related resource tuple, and changes the granted mode of the lock tuple to the requested conversion 
mode. RMi then checks whether any transaction dependencies have ceased as a result of the lock mode 
conversion. If so. RMi sends DELEDGE messages to the appropriate TMs, instructing them to delete the 
corresponding edges from the WFG. 

50 RMi then checks if other locks in the waiting queue can be granted. A lock from the head of waiting 
queue can be granted if its requested lock mode is compatible with all currently granted locks. If the lock 
can be granted then it is removed from the waiting queue, and its lock tuple is updated by setting its 
granted mode to the requested conversion mode and its status to "granted". RMi then sends a 
C0NVERTUPCOMPLETE(key,lckid,v[blk,LM GRANTED) message to the lock owner agent of each granted 

55 lock. That agent then acts as described above for the case of an upward lock conversion. 
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Cancel Lock Conversion 

When a client C requires to cancel a lock conversion, it issues the call Im cancel lock(uhandle,lckid) 

lu II lb* dyenl Ak thai represents C. 
b Wliuii Ak (eceives this call, it looks up the lock tuple in its lock pool. If it cannot find the tuple, Ickid 

must be invalid and so a LM INVLCKID status is returned to C. If the tuple is found, but no conversion is 

ponding for the lock, a LM GRANTED status is returned to C. Also if a cancel for the lock is pending, a 

LM_GnANTED status is returned to C. Otherwise Ak proceeds with the following steps. 

Ak sends a CANCEL(key, Ickid) message to the resource manager RMi that owns the lock tuple. Ak then 

w queues a ponding tuple (RMI, key, Cuhandle, message) in its pending queue and returns a Lf^ SUCCESS 

status to C. 

When RMi receives the CANCEL{key, Ickid) message from Ak, it looks up the lock tuple and related 
resource tuple. If the lock status is "granted", RMi discards the message and does not proceed with the 
foUowing siops. 

IS RMi checks whether any transaction dependencies have ceased as a result of cancelling the lock 

conversion. If so. DELEDGE messages are sent to the appropriate TMs, instructing them to remove the 
corresponding edges from the WFG. RMi then removes the lock tuple from the waiting queue and sets the 
lock status to "granted". RMi then sends a 

CONVERTUPCOMPLETE(key,lckid,vlblk,LM_CANCEL) message to the lock owner Ak. 
20 When Ak receives this message, it removes the related pending tuple from the pending queue. 

Edge Insertion 

When a resource manager RMk detects a dependency between two transactions t1 and t2, caused by 
25 the two locks II and 12. the following actions are performed. 

First. RMk selects the transaction manager TMi that holds the node t1. using the predetermined 
mapping function that maps transaction identifiers on to the transaction managers. RMk then sends an 
INSEDGE(label) message to TMi, where label = (tl ,t2.l1 ,t2), instructing it to insert an edge into the WFG. 

When TMi receives an INSEDGE(labet) message from RMk, it searches its edge pool for an edge tuple 
30 with this label value. If the edge tuple cannot be found in the edge pool, TMi constructs a new edge tuple 
with owner RMk and with this label value, and adds it to the edge pool. Otherwise TMi increments the 
cardinality of the edge tuple. 

TMi then looks up the node tuple identified by tl in its node pool. If the node tuple cannot be found, 
TMi constructs a new node tuple with cardinality equal to 1 , and adds this to the pool. The new node tuple 
35 is stamped with a timestamp that is unique within the TMs, and which determines the priority of the node. 
Otherwise TMi increments the cardinality of the node in the pool. 

If the edge cardinality is equal to one. TMi performs the following actions. First, TMi sends a PROBE- 
{init,stamp,t2) message to the transaction manager TMj that holds the node 12. The argument "init" is the 
label (tl, t2. II, 12) of the newly inserted edge. The argument "stamp" is the timestamp of the node tl. TMi 
40 then looks up all probe tuples in the probe pool whose node value is equal to tl. For each tuple found, a 
PROBE(init, stamp, t2) message is sent to the transaction manager TMj that holds the node t2, where "init" is 

the init edge label of the probe tuple, and "stamp" is the timestamp of that tuple. In other words, TMi 

forwards any stored probes to t2. 

45 Edge Deletion 

If a resource manager RMk detects that a dependency between two transactions tl and t2, caused by 
two locks II and 11, has ceased to exist, the following actions are taken. 

First, RMk selects the transaction manager TMi that holds the node corresponding to transaction tl. 
50 RMk then sends a DELEDGE(label) message to TMi, where label = (tl ,t2,li,l2). 

When TMi receives a DELEDGE(label) message from RMk, it looks up the edge tuple identified by the 
argument "label" in its edge pool and subtracts one from its cardinality. If the cardinality has become zero, 
the edge tuple is removed from the pool. 

TMi then looks up the node tuple identified by tl in its node pool and subtracts one from its cardinality. 
55 If the node cardinality has become zero, the node tuple is removed from the pool. 

If the edge cardinality is equal to zero, TMi now performs the following actions. First, TMi sends an 
ANTIPROBE(init.stamp,t2) message to the transaction manager TMj that holds the node corresponding to 
t2. The argument "init" is the label (tl, t2. II, 12) of the deleted edge. The argument "stamp" is the 

12 
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timestamp of the node ti . TMi then looks up all probe tuples in the probe pool whose node value is equal 
to ti. For each tuple found, an 

ANTIPR0BE(init,stamp,t2) message is send to the transaction manager TMj that holds the node cor- 
responding to 12, where "inrt" is the init edge label of the probe tuple, and "stamp" is the timestamp of 

5 that tuple. In other words, TMi forwards any stored antiprobes to 12. 

WFG Probe 

When a transaction manager TMi receives a PROBE{init,stamp,t) message from another transaction 
10 manager TMk, where init = (ti ,t2,l1 ,12), it performs the following actions. 

First, TMi looks up the node tuple identified by t In its node pool. If the node cannot be found, a node 
tuple is constructed and added to the pool. Otherwise the cardinality of the node tuple is incremented by 
one. 

TMi then searches its probe pool for a probe tuple whose init edge value is equal to the init parameter 

15 in the probe message. If the probe tuple cannot be found, a probe tuple is constructed with cardinality 
equal to 1 , and added to the pool. Otherwise the cardinality of the probe tuple in the pool is incremented by 
one. 

TMi then checks for a deadlock cycle. If values t and ti in the probe are equal, the probe has arrived 
back at its starting point. TMi looks up the edge in its edge pool and if the edge still exists a deadlock has 

20 been found. The probe is not further forwarded and TMi sends a DEADLOCK(init) message to the resource 
manager RMx that owns the edge. 

If deadlock is not detected, TMi proceeds with the following steps. If the probe's cardinality is equal to 
one, TMi checks whether the timestamp of the probe is greater than the timestamp of node t. If so. TMi 
forwards the probe afong all that node's outgoing edges. Thus, TMi looks up alt edges e = (tatl.head.ll .12) 

25 in the edge pool where the tail is equal to t, and sends a PROBE(tnlt,stamp.head) to the transaction 
manager TMj that holds the node corresponding to the transaction identified by the parameter "head" in the 
probe. 

WFG Antiprobe 

30 

When a transaction manager TMi receives an 
ANTIPROBE(init,stamp.t) message from another transaction manager TMj, where init = (t1,t2,l1,l2), the 
following actions are performed. 

First, TMi looks up the node tuple identified by t in its node pool and subtracts one from the cardinality 
35 of the node tuple. If the cardinality becomes zero, the node tuple is removed from the node pool. 

TMi then looks up the probe tuple in its probe pool whose init edge value is equal to the "init" 

parameter in the antiprobe, and subtracts one from the cardinality of the probe tuple. If the cardinality 
becomes zero, the probe tuple is removed from the probe pool. 

TMi then checks whether the values t and t2 in the antiprobe are equal. If so, the antiprobe has arrived 
40 back at Its starting point and is not further forwarded. 

If the cardinality of the probe tuple became zero, TMi checks whether the timestamp in the antiprobe is 
greater than the timestamp of node t. If so, TMi forwards the antiprobe along all that node's outgoing edges. 
Thus, TMI looks up all edges e = (tail, head, 11 ,12) in the edge pool where "tail" is equal to t, and sends an 
ANTIPROBE(init,stamp.head) to the transaction manager TMj that holds the node corresponding to the head 
45 transaction of that edge. 

Deadlock Resolution 

When a transaction manager TM] detects a deadlock, it generates a deadlock message DEADLOCK- 
50 (edge), where edge = (t1,t2,l1,l2). The deadlock message is sent to the resource manager RMi that owns 
the edge. 

When RMi receives a deadlock message, it looks up the lock tuple identified by 11. If the lock tuple 
cannot be found or if the lock status is granted, then the deadlock message is discarded and RMi does not 
proceed with the following steps. 
55 RMi selects the lock 11 as the victim to break the deadlock. RMi cancels the outstanding lock 

conversion for lock 11 and sends a CONVERTUPCOMPLETE(l1 ,vlblk,LM_DEADLOCK) message to the 
agent Ak that owns the lock tuple. 
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Failures 

The failure of an agent or server is detected through the closure of a connection. The surviving servers 
and agents then each take steps to recover from this failure, as will now be described. 

5 

PM Failure 

When one of the process managers PMk fails, information on processes is lost. The following steps are 
performed to recover from this. 
10 First, a replacement process manager PMk is created or selected. 

Each agent Ai then searches its agent process pool for process tuples whose owner is PMk, and sends 
these tuples to PMk. Each agent At then searches its pending queue for pending requests whose owner is 
PMk and retransmits them to PMk. The requests are sent in the order they are stored in the pending queue. 
When PMk receives the process tuples from Ai, it adds them to its process pool. Processing of the 
75 retransmitted requests is deferred until all agents have sent their process tuples. 

SM Failure 

When one of the session managers SMk fails, information on sessions is lost. The following steps are 
20 performed to recover from this. 

First, a replacement session manager SMk is created or selected. 

Each agent Ai then searches its agent session table for session tuples whose owner is SMk. and sends 
them to SMk. Each agent Ai then searches its pending queue for pending requests whose owner is SMk. 
and retransmits them to SMk. The requests are sent in the order they are stored in the pending queue. 
25 When SMk receives the session tuples from Ai, it adds them to its session pool. Processing of the 
retransmitted requests Is deferred until all agents have send their session tuples. 

RM Failure 

30 When a resource manager RMk fails, the following actions are taken to recover from this failure. 
First, a replacement resource manager RMk is created or selected; 

Each transaction manager TMi then searches its edge pool and deletes all edge tuples that are owned 
by RMk. Antiprobe messages are sent as described above. This returns the RMs and TMs to a mutually 
consistent state. 

35 Each agent Ai then searches its lock pool for lock tuples owned by RMk, and sends them to RMk. In 

other words, the agents re-send the granted lock states to RMk. When RMk receives these lock tuples, it 
adds them to its lock pool. RMk then reconstructs the resource and transaction tuples in its resource and 
transaction pools from the lock tuples, each lock being added to the lock set of the related resource and 
transaction tuple. This brings the agents and the RMs into a mutually consistent state. 

40 Each agent Ai then searches its pending queue for lock conversion cancel requests previously sent to 
RMk. If any is found, it is removed from the pending queue and a "conversion complete" event with a 
"cancel" status is posted on the client's event queue. 

Each agent Ai then searches its pending queue for pending requests whose owner is RMk, and re- 
sends them to RMk. Processing of these requests is deferred until all agents have sent their lock tuples. 

45 This restores locks in the waiting queue. The original order in which the waiting locks were queued has 
been lost, but this information is in fact irrelevant. Therefore the waiting locks may be restored in a different 
order and waiting locks may even become granted when their lock mode is compatible with the other 
granted locks. Any dependencies that are detected between locks of transactions will cause INSEDGE 
messages to be sent to the TMs, instructing them to insert edges in the WFG. 

50 

TM Failure 

If a transaction manager TMk falls, the following actions are taken to recover from the failure. 

First, a replacement transaction manager TMk is created or selected. 
55 Each other transaction manager TMi then searches its probe pool for probes previously sent to TMk. 

and re-sends these probes to TMk. Each other transaction manager TMi also searches its probe pool for 
probes owned by TMk, and for each such probe, generates an antiprobe. This brings the TMs back into a 
mutually consistent state. Note that any probes that arrive in TMk are not further forwarded since TMk does 
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not yet contain any edges along which these probes can be fon/varded. 

Independently, in parallel with the above action of the TMs, each resource manager RMI checks for any 
dependencies between the locks in its lock pool and re-sends any resulting WFG edges to TMk. These 
edges may cause further probes to be generated. 
5 Because the above steps are executed independently and in parallel, an RMI may start re-sending 

edges to TMk while other TMs are generating antiprobes for the lost edges, or are re-sending probes to 
TMk. The TMs and RMs do not require any global coordinator to coordinate the sequence of recovery 
steps. 

70 Agent Failure 

If an agent Ak fails, the following actions are taken to recover from the failure. 

Each resource manager RMi searches its lock pool and deletes all lock tuples that are owned by Ak. 
Each session manager SMi searches its session pool and deletes all session tuples that are owned by 

75 Ak. 

Each process manager PMi searches its process pool and deletes all process tuples that are owned by 

Ak. 

This brings all the agents, RMs. SMs and PMs into a mutually consistent state. The deletion of locks of 
agent Ak may result in dependency changes, and these are sent to the TMs as described above, which 
20 may cause further probe and antiprobe messages to be generated. 

Client Failure 

When an agent detects the failure of one of its clients, it removes all information about the client. The 
25 agent deletes all processes that are created by the client, closes all sessions that are opened by these 
processes and closes all locks that are opened by these sessions. The agent looks up all pending requests 
of the client. The client member in the pending tuple Is set to the reserved value FAILED. c 

Whenever an agent receives a reply from any of the servers PM, SM, or RM the agent looks up the 
corresponding pending tuple in the pending queue, and checks whether the client member of this tuple has 
30 the value FAILED. If the value is not equal to FAILED, then an event may be posted on the client's event 
queue. Otherwise the client has failed and the agent does whatever is appropriate for the reply from the 
server. Possible replies from a server are a new process, an open session, an open lock, or a convert lock 
completion message. If a message is a reply on a process creation, an open session or an open lock, then 
the agent sends a delete process, a close session or a close lock message respectively to the server in 
35 question. Other messages are discarded by the agent. In alt cases the related pending tuples are removed 
from the pending queue. 

In summary, it can be seen that resilience to failure of the servers PM, SM and RM is provided by 
duplicating information between the servers and the agents (which are the users of these servers) rather 
than by duplicating the servers. Thus, if a server fails, the information in that server can be reconstructed 
40 from the information held in the agents. Similarly, resilience to failure of the servers TM is provided by 
duplicating information between those servers and the servers RM (which are the users of the servers TM). 
in this way. resilience can be achieved without any increase in the number of messages between users and 
servers in normal running. 

45 Claims 

1. A distributed data processing system comprising a plurality of processing units wherein the processing 
units contain: 

(a) a plurality of resources, 

50 (b) a plurality of user processes performing transactions which require access to the resources, and 

for producing lock requests, requesting locks on the resources for said transactions, 
(c) a resource manager responsive to said lock requests from the user processes, for maintaining a 
queue of lock requests that cannot be immediately granted, and for detecting dependencies between 
transactions caused by conflicting lock requests, 

55 (d) a transaction manager for storing a wait-for graph comprising a plurality of nodes representing 

transactions and a plurality of edges interconnecting the nodes and representing said dependencies 
between the transactions, each edge being labelled with the identities of the lock requests that 
caused the dependency, and 
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(e) means for detecting a cyclic chain of dependencies in the wait-for graph and. upon detection of a 
cyclic chain of dependencies, for sending a deadlock nnessage to the resource manager identifying 
a particular lock request as a victim for deletion to resolve the deadlock. 

5 2. A system according to Claim 1 wherein the means for detecting cyclic dependencies comprises means 
for propagating probe messages through the wait-for graph. 

3. A system according to Claim 2 wherein the means for detecting cyclic dependencies further includes 
means for storing a copy of each probe message in each node to which it is propagated, along with a 

TO count value, and wherein if a probe message is received at a node which already contains a copy of 

that probe message, the count is incremented and the probe message is not propagated any further. 

4. A system according to Claim 1 wherein: 

a) said transaction manager comprises a plurality of transaction manager processes, each of which 
'5 holds a predetermined portion of the wait-for graph, said transaction manager processes comprising 

means for propagating probe messages between said transaction manager processes to detect 
cyclic dependencies, and 

b) said resource manager comprises a plurality of resource manager processes each of which 
includes means for storing a plurality of locks on a resource, means for checking for dependencies 

20 between said locks, and means for sending insert edge and delete edge messages to the 

transaction manager processes instructing them to insert and delete edges in the wait-for graph 
corresponding to said dependencies. 

5. A system according to Claim 4 wherein the system further includes means for allocating a replacement 
25 transaction manager process to replace a failed transaction manager process and wherein each 

resource manager process includes means for resending its insert edge messages to the replacement 
transaction manager process. 

6. A system according to Claim 5 wherein each transaction manager process includes means for storing 
30 probe messages it has sent to and received from other transaction manager processes and wherein 

each transaction manager process includes means for deleting stored probe messages received from 
said failed transaction manager process and for re-sending probe messages to said replacement 
transaction manager process. 

35 7. A system according to Claim 4,5 or 6 further including means for allocating a replacement resource 
manager process to replace a failed resource manager process, and wherein the transaction manager 
processes include means for deleting any edges associated with the failed resource manager process 
from their respective portions of the wait-for graph. 

40 8. A system according to Claim 7 wherein said user processes further include means for re-sending their - 
lock requests to said replacement resource manager process. 

9. A distributed data processing system comprising: a plurality of agents, and a plurality of servers for 
controlling access by the agents to shared resources, wherein each agent comprises means for 
45 sending messages to the servers, requesting the servers to perform access management operations. 

and means for storing a record of access management information in those messages, and wherein the 
system further includes means for allocating a replacement server to replace a failed server and for 
causing each agent to send its stored record of access management information to the replacement 
server to re-create access management information in said replacement server. 

50 
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Fig.1. 
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Fig. 2. 
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Fig. 4. 
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